WHO's Community-Centered Epidemic and Pandemic Information Platform: Hive.

Each epidemic and pandemic is accompanied by an infodemic. The infodemic during the COVID-19 pandemic was unprecedented. Accessing accurate information was difficult and misinformation harmed the pandemic response, the health of individuals and trust in science, governments and societies. WHO is building a community-centered information platform, the Hive, to deliver on the vision of ensuring that all people everywhere have access to the right information, at the right time, in the right format in order to make decisions to protect their health and the health of others. The platform provides access to credible information, a safe space for knowledge-sharing, discussion, and collaborating with others, and a forum to crowdsource solutions to problems. The platform is equipped with many collaboration features, including instant chats, event management, and data analytics tools to generate insights. The Hive platform is an innovative minimum viable product (MVP) that seeks to leverage the complex information ecosystem and the invaluable role communities play to share and access trustworthy health information during epidemics and pandemics.

Narrative Trends over the COVID-19 Pandemic: Digital Social Listening to Inform WHO Infodemic Management.

The COVID-19 infodemic is an overwhelming amount of information that has challenged pandemic communication and epidemic response. WHO has produced weekly infodemic insights reports to identify questions, concerns, information voids expressed and experienced by people online. Publicly available data was collected and categorized to a public health taxonomy to enable thematic analysis. Analysis showed three key periods of narrative volume peaks. Understanding how conversations change over time can help inform future infodemic preparedness and prevention planning.

An Adaptive Digital Intelligence System to Support Infodemic Management: The WHO EARS Platform.

The WHO Early AI-Supported Response with Social Listening (EARS) platform was developed to help inform infodemic response during the COVID-19 pandemic. There was continual monitoring and evaluation of the platform and feedback from end-users was sought on a continual basis. Iterations were made to the platform in response to user needs, including the introduction of new languages and countries, and additional features to better enable more fine-grained and rapid analysis and reporting. The platform demonstrates how a scalable, adaptable system can be iterated upon to continue to support those working in emergency preparedness and response.

Measuring the Burden of Infodemics: Summary of the Methods and Results of the Fifth WHO Infodemic Management Conference.

Background: An infodemic is excess information, including false or misleading information, that spreads in digital and physical environments during a public health emergency. The COVID-19 pandemic has been accompanied by an unprecedented global infodemic that has led to confusion about the benefits of medical and public health interventions, with substantial impact on risk-taking and health-seeking behaviors, eroding trust in health authorities and compromising the effectiveness of public health responses and policies. Standardized measures are needed to quantify the harmful impacts of the infodemic in a systematic and methodologically robust manner, as well as harmonizing highly divergent approaches currently explored for this purpose. This can serve as a foundation for a systematic, evidence-based approach to monitoring, identifying, and mitigating future infodemic harms in emergency preparedness and prevention. Objective: In this paper, we summarize the Fifth World Health Organization (WHO) Infodemic Management Conference structure, proceedings, outcomes, and proposed actions seeking to identify the interdisciplinary approaches and frameworks needed to enable the measurement of the burden of infodemics. Methods: An iterative human-centered design (HCD) approach and concept mapping were used to facilitate focused discussions and allow for the generation of actionable outcomes and recommendations. The discussions included 86 participants representing diverse scientific disciplines and health authorities from 28 countries across all WHO regions, along with observers from civil society and global public health-implementing partners. A thematic map capturing the concepts matching the key contributing factors to the public health burden of infodemics was used throughout the conference to frame and contextualize discussions. Five key areas for immediate action were identified. Results: The 5 key areas for the development of metrics to assess the burden of infodemics and associated interventions included (1) developing standardized definitions and ensuring the adoption thereof; (2) improving the map of concepts influencing the burden of infodemics; (3) conducting a review of evidence, tools, and data sources; (4) setting up a technical working group; and (5) addressing immediate priorities for postpandemic recovery and resilience building. The summary report consolidated group input toward a common vocabulary with standardized terms, concepts, study designs, measures, and tools to estimate the burden of infodemics and the effectiveness of infodemic management interventions. Conclusions: Standardizing measurement is the basis for documenting the burden of infodemics on health systems and population health during emergencies. Investment is needed into the development of practical, affordable, evidence-based, and systematic methods that are legally and ethically balanced for monitoring infodemics; generating diagnostics, infodemic insights, and recommendations; and developing interventions, action-oriented guidance, policies, support options, mechanisms, and tools for infodemic managers and emergency program managers.

The Good Talk! A Serious Game to Boost People's Competence to Have Open Conversations About COVID-19: Protocol for a Randomized Controlled Trial.

BACKGROUND: Vaccine hesitancy is one of the many factors impeding efforts to control the COVID-19 pandemic. Exacerbated by the COVID-19 infodemic, misinformation has undermined public trust in vaccination, led to greater polarization, and resulted in a high social cost where close social relationships have experienced conflict or disagreements about the public health response. OBJECTIVE: The purpose of this paper is to describe the theory behind the development of a digital behavioral science intervention-The Good Talk!-designed to target vaccine-hesitant individuals through their close contacts (eg, family, friends, and colleagues) and to describe the methodology of a research study to evaluate its efficacy. METHODS: The Good Talk! uses an educational serious game approach to boost the skills and competences of vaccine advocates to have open conversations about COVID-19 with their close contacts who are vaccine hesitant. The game teaches vaccine advocates evidence-based open conversation skills to help them speak with individuals who have opposing points of view or who may ascribe to nonscientifically supported beliefs while retaining trust, identifying common ground, and fostering acceptance and respect of divergent views. The game is currently under development and will be available on the web, free to access for participants worldwide, and accompanied by a promotional campaign to recruit participants through social media channels. This protocol describes the methodology for a randomized controlled trial that will compare participants who play The Good Talk! game with a control group that plays the widely known noneducational game Tetris. The study will evaluate a participant's open conversation skills, self-efficacy, and behavioral intentions to have an open conversation with a vaccine-hesitant individual both before and after game play. RESULTS: Recruitment will commence in early 2023 and will cease once 450 participants complete the study (225 per group). The primary outcome is improvement in open conversation skills. Secondary outcomes are self-efficacy and behavioral intentions to have an open conversation with a vaccine-hesitant individual. Exploratory analyses will examine the effect of the game on implementation intentions as well as potential covariates or subgroup differences based on sociodemographic information or previous experiences with COVID-19 vaccination conversations. CONCLUSIONS: The outcome of the project is to promote more open conversations regarding COVID-19 vaccination. We hope that our approach will encourage more governments and public health experts to engage in their mission to reach their citizens directly with digital health solutions and to consider such interventions as an important tool in infodemic management. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/40753.

WHO competency framework for health authorities and institutions to manage infodemics: its development and features.

BACKGROUND: In April 2020, the World Health Organization (WHO) Information Network for Epidemics produced an agenda for managing the COVID-19 infodemic. "Infodemic" refers to the overabundance of information-including mis- and disinformation. In this agenda it was pointed out the need to create a competency framework for infodemic management (IM). This framework was released by WHO on 20th September 2021. This paper presents the WHO framework for IM by highlighting the different investigative steps behind its development. METHODS: The framework was built through three steps. Step 1 included the preparatory work following the guidelines in the Guide to writing Competency Framework for WHO Academy courses. Step 2 was based on a qualitative study with participants (N = 25), identified worldwide on the basis of their academic background in relevant fields of IM or of their professional experience in IM activities at the institutional level. The interviews were conducted online between December 2020 and January 2021, they were video-recorded and analyzed using thematic analysis. In Step 3, two stakeholder panels were conducted to revise the framework. RESULTS: The competency framework contains four primary domains, each of which comprised main activities, related tasks, and knowledge and skills. It identifies competencies to manage and monitor infodemics, to design, conduct and evaluate appropriate interventions, as well as to strengthen health systems. Its main purpose is to assist institutions in reinforcing their IM capacities and implementing effective IM processes and actions according to their individual contexts and resources. CONCLUSION: The competency framework is not intended to be a regulatory document nor a training curriculum. As a WHO initiative, it serves as a reference tool to be applied according to local priorities and needs within the different countries. This framework can assist institutions in strengthening IM capacity by hiring, staff development, and human resources planning.

Infodemics: A new challenge for public health.

The COVID-19 information epidemic, or "infodemic," demonstrates how unlimited access to information may confuse and influence behaviors during a health emergency. However, the study of infodemics is relatively new, and little is known about their relationship with epidemics management. Here, we discuss unresolved issues and propose research directions to enhance preparedness for future health crises.

A Public Health Research Agenda for Managing Infodemics: Methods and Results of the First WHO Infodemiology Conference.

BACKGROUND: An infodemic is an overflow of information of varying quality that surges across digital and physical environments during an acute public health event. It leads to confusion, risk-taking, and behaviors that can harm health and lead to erosion of trust in health authorities and public health responses. Owing to the global scale and high stakes of the health emergency, responding to the infodemic related to the pandemic is particularly urgent. Building on diverse research disciplines and expanding the discipline of infodemiology, more evidence-based interventions are needed to design infodemic management interventions and tools and implement them by health emergency responders. OBJECTIVE: The World Health Organization organized the first global infodemiology conference, entirely online, during June and July 2020, with a follow-up process from August to October 2020, to review current multidisciplinary evidence, interventions, and practices that can be applied to the COVID-19 infodemic response. This resulted in the creation of a public health research agenda for managing infodemics. METHODS: As part of the conference, a structured expert judgment synthesis method was used to formulate a public health research agenda. A total of 110 participants represented diverse scientific disciplines from over 35 countries and global public health implementing partners. The conference used a laddered discussion sprint methodology by rotating participant teams, and a managed follow-up process was used to assemble a research agenda based on the discussion and structured expert feedback. This resulted in a five-workstream frame of the research agenda for infodemic management and 166 suggested research questions. The participants then ranked the questions for feasibility and expected public health impact. The expert consensus was summarized in a public health research agenda that included a list of priority research questions. RESULTS: The public health research agenda for infodemic management has five workstreams: (1) measuring and continuously monitoring the impact of infodemics during health emergencies; (2) detecting signals and understanding the spread and risk of infodemics; (3) responding and deploying interventions that mitigate and protect against infodemics and their harmful effects; (4) evaluating infodemic interventions and strengthening the resilience of individuals and communities to infodemics; and (5) promoting the development, adaptation, and application of interventions and toolkits for infodemic management. Each workstream identifies research questions and highlights 49 high priority research questions. CONCLUSIONS: Public health authorities need to develop, validate, implement, and adapt tools and interventions for managing infodemics in acute public health events in ways that are appropriate for their countries and contexts. Infodemiology provides a scientific foundation to make this possible. This research agenda proposes a structured framework for targeted investment for the scientific community, policy makers, implementing organizations, and other stakeholders to consider.

Infodemic Signal Detection During the COVID-19 Pandemic: Development of a Methodology for Identifying Potential Information Voids in Online Conversations.

BACKGROUND: The COVID-19 pandemic has been accompanied by an infodemic: excess information, including false or misleading information, in digital and physical environments during an acute public health event. This infodemic is leading to confusion and risk-taking behaviors that can be harmful to health, as well as to mistrust in health authorities and public health responses. The World Health Organization (WHO) is working to develop tools to provide an evidence-based response to the infodemic, enabling prioritization of health response activities. OBJECTIVE: In this work, we aimed to develop a practical, structured approach to identify narratives in public online conversations on social media platforms where concerns or confusion exist or where narratives are gaining traction, thus providing actionable data to help the WHO prioritize its response efforts to address the COVID-19 infodemic. METHODS: We developed a taxonomy to filter global public conversations in English and French related to COVID-19 on social media into 5 categories with 35 subcategories. The taxonomy and its implementation were validated for retrieval precision and recall, and they were reviewed and adapted as language about the pandemic in online conversations changed over time. The aggregated data for each subcategory were analyzed on a weekly basis by volume, velocity, and presence of questions to detect signals of information voids with potential for confusion or where mis- or disinformation may thrive. A human analyst reviewed and identified potential information voids and sources of confusion, and quantitative data were used to provide insights on emerging narratives, influencers, and public reactions to COVID-19-related topics. RESULTS: A COVID-19 public health social listening taxonomy was developed, validated, and applied to filter relevant content for more focused analysis. A weekly analysis of public online conversations since March 23, 2020, enabled quantification of shifting interests in public health-related topics concerning the pandemic, and the analysis demonstrated recurring voids of verified health information. This approach therefore focuses on the detection of infodemic signals to generate actionable insights to rapidly inform decision-making for a more targeted and adaptive response, including risk communication. CONCLUSIONS: This approach has been successfully applied to identify and analyze infodemic signals, particularly information voids, to inform the COVID-19 pandemic response. More broadly, the results have demonstrated the importance of ongoing monitoring and analysis of public online conversations, as information voids frequently recur and narratives shift over time. The approach is being piloted in individual countries and WHO regions to generate localized insights and actions; meanwhile, a pilot of an artificial intelligence-based social listening platform is using this taxonomy to aggregate and compare online conversations across 20 countries. Beyond the COVID-19 pandemic, the taxonomy and methodology may be adapted for fast deployment in future public health events, and they could form the basis of a routine social listening program for health preparedness and response planning.

EARS - A WHO Platform for AI-Supported Real-Time Online Social Listening of COVID-19 Conversations.

As the COVID-19 pandemic evolves, the accompanying infodemic is being amplified through social media and has challenged effective response. The WHO Early AI-supported Response with Social Listening (EARS) is a platform that summarizes real-time information about how people are talking about COVID-19 in public spaces online in 20 pilot countries and in four languages. The aim of the platform is to better integrate social listening with other data sources and analyses that can inform infodemic response.

WHO Digital Intelligence Analysis for Tracking Narratives and Information Voids in the COVID-19 Infodemic.

The COVID-19 pandemic is the first to unfold in the highly digitalized society of the 21st century and is therefore the first pandemic to benefit from and be threatened by a thriving real-time digital information ecosystem. For this reason, the response to the infodemic required development of a public health social listening taxonomy, a structure that can simplify the chaotic information ecosystem to enable an adaptable monitoring infrastructure that detects signals of fertile ground for misinformation and guides trusted sources of verified information to fill in information voids in a timely manner. A weekly analysis of public online conversations since 23 March 2020 has enabled the quantification of running shifts of public interest in public health-related topics concerning the pandemic and has demonstrated the frequent resumption of information voids relevant for public health interventions and risk communication in an emergency response setting.

PIPDeploy: Development and implementation of a gamified table top simulation exercise to strengthen national pandemic vaccine preparedness and readiness.

Successful emergency vaccination campaigns rely on effective deployment and vaccination plans. This applies to localised outbreaks as well as for pandemics. In the wake of the 2009 H1N1 influenza pandemic, analysis of the global Vaccine Deployment Initiative, through which the World Health Organization (WHO) donated pandemic influenza vaccines to countries in need, revealed that an absence of vaccine deployment plans in many countries significantly hindered vaccine deployment. Through the Pandemic Influenza Preparedness Framework adopted by the World Health Assembly in 2011, WHO is engaging in several capacity building activities to improve pandemic influenza preparedness and response and make provisions for access to vaccines and sharing of other benefits. The Framework calls for the development and exercise of operational plans for deployment of influenza vaccines to enhance pandemic preparedness. To this end, WHO has supported the development of PIPDeploy, an interactive, in-person table top simulation exercise to facilitate learning for emergency preparedness. It employs various game design elements including a game board, time pressure, leaderboards and teams to enhance participants' motivation. PIPDeploy formed part of five WHO Pandemic Influenza Vaccine Deployment Workshops attended by national-level managers responsible for pandemic influenza vaccine response predominantly in non-producing countries. The purpose of this study was to describe the features and application of PIPDeploy, and present findings of the evaluation of participants' experiences during the simulation involving a "hot wash" discussion and collection of quantitative data. The simulation's instructional approach was widely accepted by participants, who reported that the format was novel and engaging. They reflected on its utility for identifying gaps in their own vaccine deployment plans and regulatory frameworks for importation of vaccine products. All participants found the simulation relevant to their professional objectives. A range of other potential applications were suggested, including PIPDeploy's adaptation to sub-national contexts and to other epidemic diseases.

Artificial intelligence cooperation to support the global response to COVID-19

In an unprecedented effort of scientific collaboration, researchers across fields are racing to support the response to COVID-19. Making a global impact with AI tools will require scalable approaches for data, model and code sharing;adapting applications to local contexts;and cooperation across borders.

Framework for Managing the COVID-19 Infodemic: Methods and Results of an Online, Crowdsourced WHO Technical Consultation.

BACKGROUND: An infodemic is an overabundance of information-some accurate and some not-that occurs during an epidemic. In a similar manner to an epidemic, it spreads between humans via digital and physical information systems. It makes it hard for people to find trustworthy sources and reliable guidance when they need it. OBJECTIVE: A World Health Organization (WHO) technical consultation on responding to the infodemic related to the coronavirus disease (COVID-19) pandemic was held, entirely online, to crowdsource suggested actions for a framework for infodemic management. METHODS: A group of policy makers, public health professionals, researchers, students, and other concerned stakeholders was joined by representatives of the media, social media platforms, various private sector organizations, and civil society to suggest and discuss actions for all parts of society, and multiple related professional and scientific disciplines, methods, and technologies. A total of 594 ideas for actions were crowdsourced online during the discussions and consolidated into suggestions for an infodemic management framework. RESULTS: The analysis team distilled the suggestions into a set of 50 proposed actions for a framework for managing infodemics in health emergencies. The consultation revealed six policy implications to consider. First, interventions and messages must be based on science and evidence, and must reach citizens and enable them to make informed decisions on how to protect themselves and their communities in a health emergency. Second, knowledge should be translated into actionable behavior-change messages, presented in ways that are understood by and accessible to all individuals in all parts of all societies. Third, governments should reach out to key communities to ensure their concerns and information needs are understood, tailoring advice and messages to address the audiences they represent. Fourth, to strengthen the analysis and amplification of information impact, strategic partnerships should be formed across all sectors, including but not limited to the social media and technology sectors, academia, and civil society. Fifth, health authorities should ensure that these actions are informed by reliable information that helps them understand the circulating narratives and changes in the flow of information, questions, and misinformation in communities. Sixth, following experiences to date in responding to the COVID-19 infodemic and the lessons from other disease outbreaks, infodemic management approaches should be further developed to support preparedness and response, and to inform risk mitigation, and be enhanced through data science and sociobehavioral and other research. CONCLUSIONS: The first version of this framework proposes five action areas in which WHO Member States and actors within society can apply, according to their mandate, an infodemic management approach adapted to national contexts and practices. Responses to the COVID-19 pandemic and the related infodemic require swift, regular, systematic, and coordinated action from multiple sectors of society and government. It remains crucial that we promote trusted information and fight misinformation, thereby helping save lives.